FIG. 1 is an illustration of an exemplary conventional blade server 100 connected to an external switched fabric 108. Blade servers overcome some of the inefficiencies of individual standalone or rack-mounted One Unit (1U) high servers, each of which is self-contained and includes separate power supplies, fans, and the like, and are therefore inefficient in terms of space, power, cooling, and other characteristics. Blade servers 100 utilize a modular, plug-in approach wherein the housing for each server is eliminated along with self-contained components such as power supplies and fans. Each previously standalone server is therefore reduced to a server “blade” 102 capable of being plugged into a backplane or midplane 104 within the blade server chassis 106 from the front of the chassis. Typically, two to 14 blades may be installed in a single blade server chassis 106. The midplane 104 contains connectors for receiving the server blades 102 and typically contains from one to four “lanes” or paths on a Printed Circuit Board (PCB) for carrying high-speed data signals. The midplane 104 therefore eliminates much of the cabling that was required with individual servers. The blade server chassis 106 also provides redundant common cooling and power to the server blades 102 through the midplane 104.
Conventional blade servers 100 may be connected to redundant external switch fabrics 108 through interface cards such as an “A” side Input/Output (I/O) switch 110 and a “B” side I/O switch 112, which plug into the midplane 104 from the back of the chassis 106. Typically, the redundancy enables one switch to take over if the other fails. Alternatively, in Switched Bunch Of Disk (SBOD) or Just a Bunch Of Disks (JBOD) implementations, a blade server may comprise multiple disk drives contained in SBODs or JBODs connected to the backplane through an interface card without a connection to the network, instead of the I/O switches 110 and 112 shown in FIG. 1. In a typical SBOD, the server blades 102 have an I/O port directly on them, and talk to the drives over the midplane.
The blade server midplane 104 is typically rooted/designed to allow for one or more independent redundant fabrics or I/O protocols, such as Fibre Channel (FC), Ethernet or InfiniBand. In the case of a FC configuration, each embedded switch 110 and 112 may be a FC Arbitrated Loop (FC_AL) switch or a full fabric switch, with a separate port to connect to each of the multiple server blades 102 (but shown connected to only one server blade 102 in FIG. 1) over a FC link 116 or 118, and output ports for connecting to the external switched fabrics 108.
To enable the server blades 102 to communicate with the switch fabric, a mezzanine I/O card 114 that performs a Host Bus Adapter (HBA) (a.k.a. IOC) function is typically required in each server blade 102. These mezzanine I/O cards 114 may be mounted to the server blades 102 as daughter cards. Alternatively, an IOC may be mounted directly on the server blade. For purposes of this specification, mezzanine I/O cards 114, referred to herein, include both daughter cards and IOCs mounted directly onto the server blade. The connections to the mezzanine I/O card 114 include the two I/O links 116 and 118 routed from each of the two embedded switches 110 and 112 through the midplane 104. The mezzanine I/O cards 114 follow the standard device driver model, so that when a server blade 102 with a mezzanine I/O card 114 is plugged into the midplane 104 and connected to an embedded switch 110 or 112, it appears to be a standalone server communicating with an external switch. Note that switches 110 and 112 in FIG. 1 may alternatively be replaced with “pass-through” transceiver cards.
Depending on where the I/O switches 110 and 112 are plugged into the midplane 104, and where the server blades 102 are plugged into the midplane, the redundant data paths 116 and 118 may be drastically different in length, very similar in length, or anywhere in between. Given the conventional implementation of a blade server 100 as illustrated in FIG. 1, or alternatively in SBOD or JBOD implementations or any backplane/midplane system with redundant data paths, the different lengths will cause each of the two redundant data paths 116 and 118 to have different and distinguishable losses at all frequencies, provided impedance matching of the system is maintained to a reasonable degree. These differences may cause transmission errors and unacceptable error rates, among other things.
In some legacy blade servers, the driver circuitry generates the best signal possible, and the receiver circuitry simply receives the transmitted signals with differences in attenuation, however large or small, as influenced by the path lengths. However, because frequencies and data rates have increased, resulting in higher signal attenuation, especially at the upper (higher) frequencies of the data signal (and hence more deterministic jitter), conventional modern systems may employ transmitter circuits with pre-emphasis on their transmit outputs in an attempt to transmit data with an enhanced frequency response in order to receive a flat frequency response at the far or “receiving” end, and achieve acceptable bit error rates.
Referring to FIG. 2, a standard amplifier 200 in a transmitter circuit generates an output “eye” diagram 202 in the time domain. At lower frequencies, the transmit signal peak-to-peak amplitude may be substantial as indicated at 204, but at higher frequencies, the transmit signal peak-to-peak amplitude may be less substantial as indicated at 206, and the eye will get progressively smaller. In the frequency domain, the higher frequencies taper off in amplitude, as indicated at 208. Given a 4.250 Gbit/sec signal, at 2125 MHz the amplitude may taper off by 6 dB, half the original voltage or more. Because of the rolloff of the amplifier at high frequencies, the eye in the time domain closes at those frequencies, and data can no longer be accurately transmitted with acceptable error rates, if at all.
To compensate for this frequency domain rolloff, programmable pre-emphasis circuitry can be added to the amplifiers to give the amplifier an equivalent boost at the higher frequencies. For example, firmware and registers can be employed to control the amount of boost (de-emphasis) by communicating a 3-bit word (e.g. 000 to 111) 210 to pre-emphasis circuitry 212 within the amplifier, which is well-understood by those skilled in the art. The pre-emphasis circuitry 212 is adjustable and provides internal compensation to compensate for the path so that the frequency response at the receiving end becomes relatively flat over all frequencies (see reference character 214), which is desirable. However, even though the true definition of pre-emphasis is boosting of the higher frequencies, the transmit amplifier is limited by its supply voltages. For example, if the transmit amplifier is supplied by a 3 volt rail, the eye cannot be more than 3 volts high (at which point “rail-out” occurs), and thus even with pre-emphasis the output amplitude cannot be more than 3 volts. Therefore, rather than boost the higher frequencies to above 3 volts, which is not possible, pre-emphasis circuitry actually maximizes the output at the upper frequencies (e.g. 2125 MHz) towards the upper rail (e.g. +3V), and de-emphasizes the lower frequencies to create the relatively flat frequency response. Depending on the bandwidth of the amplifier, the amplitude of the resultant eye will roughly correspond to the amplitude at which the rolloff occurs after equalization.
Alternatively or in addition to transmit circuit pre-emphasis, conventional modern systems may also have IOC integrated circuits with equalizers on their receiver inputs, or equalizer circuits external to and ahead of the IOC integrated circuits, to customize the frequency response for a particular path and equalize the redundant data paths so that they have approximately the same frequency response over a given frequency range.
Referring now to FIG. 3, at the transmitter end 300 pre-emphasis 314 may be used as described above to de-emphasize the lower frequencies (see reference character 302) to flatten out the frequency response as seen at the receiving end. However, after the signal passes through the transmission lines 306 in the midplane of the blade server, rolloff (see reference character 308) once again occurs at the higher frequencies at the receive end 310, and the signal level may be much lower (e.g. 250 mV peak to peak), which is not near any rail. If the transmitter cannot pre-emphasize enough to substantially reduce receive end rolloff, equalization 316 may be applied to the receive end to boost the higher frequencies with gain to raise the signal level of all frequencies, and flatten out the frequency response (see reference character 312) to some predetermined level acceptable to the receive end circuitry (e.g. 600 mV peak to peak).
Ideally, the amount of equalization and/or pre-emphasis applied to each transmit amplifier or receiver would be customized for the length of the path. Some modern conventional midplanes and devices do have some form of slot identification or path length determination built into their hardware. Slot identification typically requires either dedicated pins on interface connectors or semiconductors (such as Electrically Erasable Programmable Read Only Memories (EEPROMs)) to be designed into the original Backplane/Midplane.
However, in order to save on costs, companies also desire to re-use previous designs whenever possible, especially backplanes and midplanes already in service at customer sites that do not have slot identification. Without slot identification, it is difficult to determine the proper pre-emphasis and/or equalization settings for the path loss (link budget) at the faster data rates. Too much pre-emphasis or equalization will have the same effect as too little pre-emphasis or equalization, resulting in a higher bit error rate and less than optimized performance. Therefore, when a device such as a server blade is plugged into the midplane, either human assessment and intervention is required to determine the slot location, approximate the path lengths, and apply a custom amount of pre-emphasis and/or equalization to the transmit and receive circuits, or alternatively, a fixed “compromise” amount of pre-emphasis and/or equalization for an average path length is applied that hopefully works for both paths regardless of where the device is plugged into the midplane. The former approach is often not practical, and the latter approach may result in some slots having overdriven signals and others having underdriven signals, resulting in unacceptable error rates and the need for complicated gain control circuitry.
Therefore, there is a need to automatically identify and/or estimate the lengths of data paths between redundant interface devices and one or more IOCs within the blade server to assist in equalizing the frequency response of those data paths, without requiring dedicated pins on interface connectors or semiconductors designed into the midplane.